1/1 


AD-A15B  376  ISSUES  TO  BE  CONSIDERED  IN  THE  EVALUATION  OF  TECHNICAL 
PROPOSALS  FROM  THE.  .  <U>  AIR  FORCE  URIGHT  AERONAUTICAL 
LABS  MRIGHT-PATTERSON  AFB  OH  V  L  CASTOR  IB  JUN  85 
UNCLASSIFIED  AFMAL-TR-85-11B8  F/G  9/2  NL 


AD- A 158  376 


AFWAL-TR-85-1100 


ISSUES  TO  BE  CONSIDERED  IN  THE  EVALUATION  OF 
TECHNICAL  PROPOSALS  FROM  THE  ADA  LANGUAGE 
PERSPECTIVE 


ft! 


Virginia  L.  Castor 

Support  Systems  Branch 
System  Avionics  Division 


June  1985 


Final  Report 


DT1C 

^%ELECTE 
W  AU62  1B85 

T* 


I 


Approved  for  public  release;  distribution  unlimited. 


AVIONICS  LABORATORY 

AIR  FORCE  WRIGHT  AERONAUTICAL  LABORATORIES 
AIR  FORCE  SYSTEMS  COMMAND 
WRIGHT-PATTERSON  AIR  FORCE  BASE,  OHIO  45433 


NOTICE 


When  Government  drawings,  specifications,  or  other  data  are  used  for  any 
purpose  other  than  in  connection  with  a  definitely  related  Government  procure¬ 
ment  operation,  the  United  States  Government  thereby  incurs  no  responsibility 
nor  any  obligation  whatsoever;  and  the  fact  that  the  government  may  have 
formulated,  furnished,  or  in  any  way  supplied  the  said  drawings,  specifications 
or  other  data,  is  not  to  be  regarded  by  implication  or  otherwise  as  in  any 
manner  licensing  the  holder  or  any  other  person  or  corporation,  or  conveying 
any  rights  or  permission  to  manufacture  use,  or  sell  any  patented  Invention 
that  may  in  any  way  be  related  thereto. 

This  report  has  been  reviewed  by  the  Office  of  Public  Affairs  (ASD/PA) 
and  is  releasable  to  the  National  Technical  Information  Service  (NTIS) .  At 
NTIS ,  it  will  be  available  to  the  general  public,  including  foreign  nations. 

This  technical  report  has  been  reviewed  and  is  approved  for  publication. 


Virginia  L.  Castor 


Virgl 
Project  Engineer 


Date 


FOR  THE  COMMANDER 


f ''f  ■-  j  fiOCI,  MAJ,  UDAF 

Fa  Division 
'-i:  .  LjLurutory 


/< 


/nr 


Date 


"If  your  address  has  changed,  if  you  wish  to  be  removed  from  our  mailing 
list,  or  if  the  addressee  is  no  longer  employed  by  your  organization  please 
notify  AFWAL/AAAF-2 ,  W-PAFB,  OH  45433  to  help  us  maintain  a  current  mailing 
list." 

Copies  of  this  report  should  not  be  returned  unless  return  is  required  by 
*  curity  considerations,  contractual  obligations,  or  notice  on  specific 
document . 


1«.  REPORT  SECURITY  CLASS  I F I  CAT  ION 

Unclassified 


2»  SECURITY  CLASSIFICATION  AUTHORITY 

N/A 


2b  DECLASSIFICAT I ON/OOWN  GRADING  SCHEDULE 

I  N/A 


U  PERFORMING  ORGANIZATION  REPORT  NUMBERISI 

|  AFWAL-TR-85-1100 


REPORT  DOCUMENTATION  PAGE 


lb.  RESTRICTIVE  MARKINGS 


3.  DISTRIBUTION/AVAILABILITY  OF  REPORT 

Approved  for  public  release; 
Distribution  unlimited 


5.  MONITORING  ORGANIZATION  REPORT  NUMBERISI 


6>  NAME  OF  PERFORMING  ORGANIZATION 

Air  Force  Wright 
Aeronautical  Laboratories 


6c.  ADDRESS  (City.  Slate  and  ZIP  Code I 

Wright-Patterson  AFB,  OH  45433-6543 


7a.  NAME  OF  MONITORING  ORGANIZATION 


7b.  AOORESS  ICily.  Stair  and  ZIP  Code ) 


Ba.  NAME  OF  FUNOING/SPONSORlNG 
ORGANIZATION 

Ada  Joint  Program  Office 


Sc.  AOORESS  (City,  State  and  ZIP  Code) 

3D139  (Fern  st/C107)  Pentagon 
Washington,  DC  20301 


IT  TITLE  /Include ‘Security  Classification)  iSSUGS  tO  bG 

Considered  in  the  Evaluation  of  Technical  (16 


12.  PERSONAL  AUTHOR(S) 

Castor,  Virqini 


.  OFFICE  SYMBOL  9.  PROCUREMENT  INSTRUMENT  IDENTIFICATION  NUMBER 
(If  applicable) 


10.  SOURCE  OF  FUNDING  NOS. 


PROGRAM 
ELEMENT  NO. 


63226 


1«.  DATE  OF  REPORT  (Yr..  Mo..  Day) 

1985  June  10 


PROJECT 

TASK 

NO. 

NO. 

AJP0 

28 

WORK  UNIT 
NO. 


15.  PAGE  COUNT 

27 


i«  supplementary  notation  Proposals  from  the  Ada*  Language  Perspective 
*Ada  is  a  Registered  Trademark  fo  the  U.S.  Government  (Ada  Joint  Program  Office) 


COSATI  COOES 


1 8-  SUBJECT  TERMS  ( Continue  on  reverie  if  neceuary  and  identify  by  block  number ) 

Evaluation  Programming  Support  Environments 

Methodology  Program  Design  Language 

Ada* 


19  ABSTRACT  ( Continue  on  reverie  if  neceuary  and  identify  fry  block  number / 

...-  /This  report  identifies  issues  for  consideration  in  the  assessment  of  technical 
proposals  that  are  submitted  in  response  to  a  Request  For  Proposal  (RFP)  which 
specifies  the  use  of  Ada.  Detailed  questions  are  provided  on  the  following  five  main 
technical  proposal  elements;  (1)  Methodology;  (2)  Program  Design  Language; 

(3)  Implementation  Language;  (4)  Environment;  and  (5)  Personnel.  __  ...  -  ,  . 

J  y  j 


20  DlSTRl  BUTION/AV  AILABILIT  V  OF  ABSTRACT 

UN  C  LASS  I F  IEOIUNLIMITED  [3  SAME  AS  RPT  C  OTIC  USERS  □ 


33a  NAME  OF  RESPONSIBLE  INDIVIDUAL 

Virginia  L.  Castor 


21  ABSTRACT  SECURITY  CLASSIFICATION 

Unclassified 


22b  TELEPHONE  NUMBER 

(srtraea* 


22c  OFFICE  SYMBOL 

AFWAL/AAAF-2 


PREFACE 


This  report  identifies  issues  relevant  to  the  evaluation  of  technical 
proposals  in  which  use  of  the  Ada* programming  language  is  addressed.  In 
draft  form,  this  report  was  entitled  "Guidelines  for  the  Evaluation  of 
Technical  Proposals  From  the  Ada  Perspective".  However,  because  it  did  not 
include  specific  weighted  responses  to  the  questions  presented,  the  title 
has  been  modified  to  more  accurately  reflect  the  nature  of  the  report; 
i.e.,  to  identify  many  of  the  issues  from  which  the  technical  proposal 
evaluators  may  select  and  weight  specific  criteria. 

The  author  is  grateful  to  the  following  individuals  for  their  review  and 
many  valuable  comments  on  the  draft  version  of  this  report:  M.  Bassman,  S. 
Benning,  M.  Burlakoff,  B.  Colborn,  P.  Dobbs,  B.  Edson,  J.  Foidl,  J.  Friedman, 
R.  Fritz,  K.  Gilroy,  S.  Harbaugh,  J.  Hines,  R.  Hodges,  P.  Howe,  T.  Janssen, 

D.  Jacobs,  J.  Johnson,  P.  Johnson,  J.  Kramer,  J.  Lane,  L.  Lindley,  R.  Long, 

G.  Martinez,  P.  Mauro,  C.  McDonald,  L.  McFawn,  G.  McKee,  J.  Mellby,  C. 

Miller,  S.  Minear,  D.  Morse,  H.  Romanowsky,  L.  Russell,  J.  Salasin,  T. 
Saunders,  B.  Schaar,  R.  Schmalz,  J.  Shirley,  C.  Smith,  P.  Stachour,  P. 
Sullivan,  S.  Tavan,  J.  Williamson,  S.  Wong,  J.  Wood,  and  T.  Zwiebel. 

The  author  also  extends  sincere  appreciation  to  Mr.  Marvin  Spector, 

Chief  of  the  System  Avionics  Division.  Mr.  Spector  originally  suggested  the 
idea  for  development  of  this  report  and  continued  to  provide  the 
encouragement  and  support  which  enabled  its  fruition. 


*Ada  is  a  registered  trademark  of  the  U.  S.  Government  (ADA  Joint  Program 
Office) . 


DTIC 


B 


|  Accession  For 

|  NT IS  0PA& I 
!  nTIC  IaB 

|  Ju.;*  1  (  1 

i_  . . 

i  Bv _ _ 


t 


TABLE  OF  CONTENTS 


SECTION  PAGE 


1.0  INTRODUCTION . 1-1 

2.0  ELEMENTS  OF  A  PROPOSAL . 2-1 

2.1  Methodology . 2-2 

2.2  Program  Design  Language . 2-7 

2.3  Implementation  Language . 2-10 

2.4  Environment . 2-13 

2.5  Personnel . 2-15 

3.0  SUMMARY . 3-1 

4.0  REFERENCES . 4-1 


v 


SECTION  1 


1.0  INTRODUCTION 


The  purpose  of  this  report  is  to  provide  issues  for  consideration  in  the 
assessment  of  technical  proposals  that  are  submitted  in  response  to  a  Request 
For  Proposals  (RFP)  which  specifies  the  use  of  Ada.  Regardless  of  the  main 
thrust  of  a  procurement  effort,  if  the  use  of  an  Ada-based  Program  Design 
Language  (PDL)  and/or  the  use  of  Ada  as  the  Implementation  language  is 
required,  issues  identified  from  the  Ada  perspective  can  facilitate  a  more 
comprehensive,  equitable  technical  evaluation.  This  report  is  intended  for 
use  in  the  technical  evaluation  only  and,  therefore,  does  not  address  the 
area  of  cost.  This  report  should  not  be  used  as  the  sole  reference  for 
technical  evaluations,  but  rather,  as  an  aid  to  existing  technical  proposal 
evaluation  criteria  to  provide  emphasis  on  those  technical  areas  which 
address  the  use  of  Ada. 

The  issues  for  consideration  are  provided  at  a  relatively  high  level  in 
an  attempt  to  limit  the  size  of  the  document  to  one  which  will  be  referenced 
more  readily  for  actual  evaluations.  A  list  of  references  is  provided  as  an 
assistance  to  those  who  wish  to  obtain  additional  detail  on  each  of  the  areas 
addressed  within  this  report. 

Although  the  primary  intent  of  this  report  is  for  use  in  the  technical 
evaluation  of  responses  to  an  RFP,  the  issues  which  are  identified  are  also 
useful  during  the  initial  stages  of  RFP  and/or  proposal  development.  They 
can  serve  as  a  source  for  dialogue  between  government  and  industry 
personnel  and  they  can  also  be  of  benefit  to  industries  which  seek  to 
provide  proposals  that  exhibit  the  most  comprehensive  information  on 
Ada-related  qualifications. 
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SECTION  2 


2.0  ELEMENTS  OF  A  PROPOSAL 


There  are  many  aspects  of  a  proposal  which  are  subject  to  evaluation 
however,  this  report  addresses  those  primary  technical  elements  which  are 
most  significantly  impacted  through  the  use  of  Ada.  This  section  is 
organized  as  follows:  A 

-  Section  2. 1M METHODOLOGY)^ identifies  issues  associated  with  the 
assessment  of  the  methodology  which  is  proposed  for  accomplishing 
the  software  life  cycle  tasks  specified  within  the  RFP  Statement 
of  Work.  Much  of  the  information  provided  in  this  chapter  was 
extracted  directly  from  reference  [6],  Detailed  information  on 
software  methodologies  is  also  available  from  references  [5], 

[8],  and  [21].  ^ 

-  Section  2.2^R0GRAM  DESIGN  LANGUAGE}" identifies  issues 
associated  with  the  assessment  of  the  Ada-based  PDL  which  is 
proposed  for  accomplishing  the  software  design  tasks  specified 
within  the  RFP  Statement  of  Work.  Detailed  information  on 
Ada-based  PDLs  is  available  frop^ references  [11],  [14],  [17], 

[18],  [19],  and  [20]. 

-  Section  2.3 '[IMPLEMENTATION  LANGUAGE)^ identifies  issues 
associated  with  the  assessment  of  Ada  as  the  implementation 
language,  as  well  as  the  associated  Ada  compiler  and  run  time 
support  environment  which  are  proposed  for  accomplishing  the 
software  development  tasks  specified  within  the  RFP  Statement  of 
Work. ^  Detailed  Information  on  compilers  and  run  time  support 
environments  Is  available  from  references  [1],  [2],  [10],  [13], 

[15],  and  [16]. 

i 

-  Section  2. 4^  ENVIRONMENT}-  identifies  issues  associated  with  the 
assessment  of  the  programming  support  environment  which  is 
proposed  for  accomplishing  all  of  the  tasks  specified  within  the 
RFP  Statement  of  Work.  Detailed  Information  on  environments  is 
available  from  references  [3],  [4],  [7],  [9],  and  [12], 

-  Section  2. S7 (PERSONNEL)  Identifies  Issues  associated  with  the 
assessment  of  the  qualifications  of  personnel  proposed  for 
accomplishing  all  of  the  tasks  specified  within  the  RFP  Statement 
of  Work.  Primary  emphasis  is  given  to  the  level  of  expertise  and 
training  associated  with  each  individual  with  regard  to 
methodologies,  Ada-based  PDLs,  Ada  programming,  and  programming 
support  environments. 
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2.1  METHODOLOGY 


The  methodology  which  is  proposed  for  use  in  accomplishing  the  tasks 
specified  within  the  RFP  Statement  of  Work  significantly  impacts  the  process 
by  which  those  tasks  are  accomplished.  A  proposal  which  does  not  address  the 
issue  of  methodology  is  deficient  in  one  of  the  most  important  aspects  of  the 
software  development  process.  For  those  proposals  which  do  address  the  issue 
of  methodology,  the  following  questions  will  be  useful  in  evaluating  the 
suitability  of  the  proposed  methodology  for  its  use  in  the  overall  program. 


1.  If  the  RFP  does  not  require  use  of  a  particular  methodology,  has 
a  specific  software  development  methodology  been  proposed?  If 
so,  what  supporting  rationale  (productivity  or  product  quality) 
has  been  provided  for  selection  of  this  methodology  over  other 
potential  methodologies? 

2.  What  are  the  key  concepts  or  underlying  principles  upon  which 
the  methodology  is  based? 

3.  What  phases  of  the  software  development  process  are  covered  by 
the  proposed  methodology: 

-  plans/analyses 

-  software  requirements  analysis 

-  preliminary  design 

-  detailed  design 

-  coding  and  unit  testing 

-  computer  software  component  integration  and  testing 

-  computer  software  configuration  item  testing 

-  complete  system  testing 

-  maintenance/evolution 


4.  How  does  the  proposed  methodology  ensure  a  smooth  transition 
from  one  phase  to  the  next? 

5.  Is  the  proposed  methodology  particularly  well  suited  for  use  on: 


military  sponsored  software  development  efforts,  with 
adherence  to  all  military  standards  specified  within  the 
RFP  Statement  of  Work 


-  the  software  development  application  specified  within  the 
RFP  Statement  of  Work 
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-  the  level  of  effort  specified  within  the  RFP  Statement  of 
Work,  in  terms  of  system  and  personnel  resources  required 

6.  Which  of  the  following  technical  concepts  does  the  proposed 
methodology  support? 

-  rapid  prototyping 

-  function  hierarchy/decomposition 

-  data  hierarchy/data  abstraction 

-  interface  definitions 

-  data  flow 

-  sequential  control  flow 

-  reusable  components 

-  concurrency/parallelism 

-  formal  program  verification 

-  others  (specify) 

7.  What  representation  schemes  are  used? 

-  Program  Design  Language  (PDL) 

-  flowcharts 

-  hierarchical  charts 

-  data  flow  diagrams 

-  finite  state  diagrams 

-  control  flow  diagrams 

-  decision  tables/trees 

-  entity-relationship  diagrams 

-  others  (specify) 

8.  What  work  products  are  prescribed  for  users  of  the  methodology 
and  how  are  they  defined?  Do  such  work  products  conform  to 
standards  specified  within  the  RFP  Statement  of  Work? 

-  MIL-STD-490 
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-  DOD-STD-2167 


-  others 

9.  For  each  of  the  work  products  generated,  what  procedures  are 
used  to  ensure  the  quality  of  the  product: 

-  author/reader  cycle 

-  structured  walkthroughs 

-  code  audits 

-  reviews 

-  testing  (unit,  integration,  system) 

-  formal  verification 

-  others  (specify) 

10.  Two  critical  areas  of  software  definition  are  subsystem 
interfaces  and  common  data  definition.  How  does  the  methodology 
ensure  that  the  interfaces  and  data  are  properly  defined? 

11.  How  does  the  methodology  define  the  testing  environment  criteria 
and  techniques?  How  well  is  testing  tied  to  the  requirements? 

12.  By  what  procedures  is  the  completed  software  validated  against 
the  original  requirements? 

13.  How  does  the  proposed  methodology  accommodate  modifications  to 
the  original  requirements  during  coding,  integration  and  testing 
phases?  Through  what  mechanism  does  this  methodology  track  and 
report  such  modifications? 

-  engineering  change  proposals 

-  software  change  notices 

-  others  (specify) 

14.  Does  the  proposed  methodology  specifically  address  personnel 
management: 

-  team  organizations 

-  matrix  management 

-  others  (specify) 
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15.  Does  the  proposed  methodology  specifically  address  project 
management: 

-  validated  workproducts 

-  configuration  management 

*  product  identification 

*  status  accounting 

*  configuration  audits 

*  version  control 

*  release  identification 

-  others  (specify) 

16.  Does  the  proposed  methodology  specifically  address  maintenance 
(evolution)  management?  If  so,  how? 

17.  What  automated  tools  support  the  activities  of  the  proposed 
methodology?  For  each  tool  specified.  Is  the  use  of  that  tool 
optional  or  mandatory? 

18.  What  computer  resources  are  required  specifically  to  support  the 
use  of  automated  tools?  How  will  the  proposed  tools  assist  in 
improving  quality  and/or  productivity  within  the  methodology? 

19.  In  addition  to  the  documentation  specified  within  the  RFP,  what 
additional  forms  of  documentation  are  proposed?  Does  the  format 
for  documentation  adhere  to  standards  specified  within  the  RFP 
Statement  of  Work? 

20.  What  "checks  and  balances"  (both  internally  and  externally  from 
the  Issuing  organization)  are  provided  to  ensure  conformance  to 
the  proposed  methodology? 

21.  Does  the  methodology  provide  a  means  to  assess  the  progress  of 
the  software  development? 

22.  Does  the  methodology  encourage  the  early  detection  of  errors? 

23.  What  documentation  Is  available  on  the  proposed  methodology? 

24.  Does  the  proposed  methodology  address  allocation  of  timeline 
requirements  and  management  of  computer  resource  allocation  to 
ensure  that  system  timeline  requirements  will  be  met? 
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25.  Does  the  proposed  methodology  include  the  use  of  cost  models  to 
evaluate  the  feasibility  of  the  proposed  development  manpower, 
schedule,  and  cost? 

26.  On  what  other  programs  of  comparable  level  of  effort  has  the 
proposed  methodology  been  successfully  implemented?  Are  points 
of  contact  provided  to  enable  confirmation  of  this  information? 

27.  If  Ada  is  being  used  as  a  PDL,  how  well  does  the  proposed 
methodology  support  its  use? 

28.  If  Ada  is  being  used  as  the  implementation  language,  how  well 
does  the  proposed  methodology  support  the  software  engineering 
goals  facilitated  through  use  of  the  Ada  language? 

29.  How  does  the  proposed  methodology  specifically  exploit  the  use 
of  Ada  (both  as  a  PDL  and  as  an  implementation  language)? 

30.  For  each  of  the  following  Ada  features,  how  does  the  proposed 
methodology  facilitate  mapping  the  software  requirements 
specification  and/or  design  into  Ada? 

-  packages  (data  abstraction) 

-  tasks  (concurrency) 

-  generics  (packages,  types) 

-  exception  handling 

-  machine  representation  specifications 

31.  Are  there  aspects  of  the  proposed  methodology  which  do  not 
facilitate  mapping  to  the  use  of  Ada? 

32.  How  does  the  proposed  methodology  ensure  proper  transition  from 
design  (using  an  Ada-based  PDL)  "to  implementation  (using  Ada  as 
the  implementation  language)? 

33.  How  does  the  proposed  methodology  ensure  proper  transition  from 
design  (using  an  Ada-based  PDL)  to  implementation  (using  a 
language  other  than  Ada  as  the  implementation  language)? 
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SECTION  3 


3.0  SUMMARY 


The  Issues  enumerated  in  this  report  provide  a  baseline  from  which  to 
select  those  which  are  most  appropriate  for  a  particular  procurement 
activity.  Natural  extensions  to  this  report  would  include  specific  and 
weighted  criteria  to  the  RFP  responses  based  upon  the  requirements  of  the 
application  area  and  the  overall  program  goals.  The  issues  may  also  be 
applicable  for  use  during  the  various  phases  of  the  project  itself. 

If  detailed  evaluation  criteria  are  to  be  included  in  the  RFP,  page 
restrictions  on  responses  to  the  RFP  must  be  modified  to  accommodate  the 
volume  of  information  which  will  be  required.  Such  responses  will  enable 
industry  personnel  to  more  accurately  document  their  qualifications  and 
expertise  with  regard  to  Ada.  The  end  result  will  be  a  more  detailed  and 
organized  approach  to  the  evaluation  and  a  more  comprehensive  and  equitabl 
assessment  of  the  responses  by  government  personnel. 


7.  What  informal  training  mechanisms  are  provided  to  personnel: 

-  self-help  tutorial  materials  (documentation,  video  tape, 
on-line  training) 

-  periodic  problem  solving  meetings 

8.  Have  training  needs  been  identified  and  reflected  in  the  program 
schedule? 

9.  If  the  reuse  of  existing  Ada  software  components  has  been 
proposed,  what  is  the  level  of  personnel  expertise  with  regard 
to  reuse  of  these  components? 

10.  Do  management  personnel  have  previous  experience  with  comparable 
Ada  software  development  efforts?  Are  points  of  contact  provided 
to  enable  confirmation  of  this  information? 


2-17 


-  participation  in  programs  using  Ada  as  the  implementation 
language 

*  level  of  effort  of  program 

*  specific  responsibility  within  program 

*  duration  of  program  Involvement 

4.  Are  personnel  experienced  In  the  use  of  the  proposed 
environment?  If  so,  what  specific  qualifications  are  provided 
for  the  proposed  individuals: 

-  formal  training  (source,  content,  duration) 

-  in-house  training  (source,  content,  duration) 

-  participation  in  programs  using  the  proposed  environment 

*  level  of  effort  of  program 

*  specific  responsibility  within  program 

*  duration  of  program  involvement 

5.  Does  the  contractor  propose  to  obtain  outside  or  provide 
in-house  professional  training  to  personnel  on: 

-  the  proposed  methodology 

-  application  of  the  proposed  Ada-based  PDL 

-  application  of  Ada  as  the  implementation  language 

-  use  of  the  proposed  environment 

6.  What  is  the  estimated  training  period  required  for  new  personnel 
to  learn: 

-  the  proposed  methodology 

-  application  of  the  proposed  Ada-based  PDL 

-  application  of  Ada  as  the  implementation  language 

-  use  of  the  proposed  environment 


2.5  PERSONNEL 


The  cadre  of  people  proposed  to  accomplish  a  software  development 
program  consists  of  individuals  of  varying  levels  of  expertise  and  training. 
The  specification  of  the  qualifications  of  such  individuals  is  sometimes 
misleading,  with  little  factual  data  to  substantiate  high  level  claims  of 
expertise.  While  the  requirements  of  personnel  qualifications  will  differ 
by  necessity  due  to  the  nature  of  the  application,  the  following  questions 
will  be  useful  in  evaluating  the  suitability  of  the  qualifications  of  the 
individuals  proposed  for  the  overall  program. 

1.  Are  personnel  experienced  in  the  use  of  the  proposed 
methodology?  If  so,  what  specific  qualifications  are  provided 
for  the  proposed  individuals: 

-  formal  training  (source,  content,  duration) 

-  in-house  training  (source,  content,  duration) 

-  participation  in  programs  using  this  methodology 

*  level  of  effort  of  program 

*  specific  responsibility  within  program 

*  duration  of  program  Involvement 

2.  Are  personnel  experienced  In  the  use  of  the  proposed  Ada  PDL?  If 
so,  what  specific  qualifications  are  provided  for  the  proposed 
individuals: 

-  formal  training  (source,  content,  duration) 

-  in-house  training  (source,  content,  duration) 

-  participation  In  programs  using  this  Ada  PDL 

*  level  of  effort  of  program 

*  specific  responsibility  within  program 

*  duration  of  program  involvement 

3.  Are  personnel  experienced  in  the  use  of  Ada  as  an  implementation 
language?  If  so,  what  specific  qualifications  are  provided  for 
the  proposed  Individuals: 

-  formal  training  (source,  content,  duration) 

-  In-house  training  (source,  content,  duration) 


9.  What  performance  measurement  Information  (efficiency,  usability) 
is  provided  with  regard  to  the  tools? 

10.  What  system  resources  (cpu,  memory,  peripherals)  are  required 
for  use  of  the  proposed  environment? 

11.  Are  available  host  resources  adequate  for  efficient  use  of  the 
tools? 

12.  If  the  host  and  target  systems  are  different  machines,  what 
facilities  are  provided  to  enable  interactive  debugging  and 
real-time  monitoring  of  software  execution? 

13.  Will  the  development  or  acquisition  of  new  software  tools  be 
required  in  order  to  support  the  tasks  specified  within  the  RFP 
Statement  of  Work?  If  so,  have  the  rationale  for  such  tools  and 
the  availability  schedules  been  provided? 

14.  Has  program  integrity  been  addressed  in  the  transfer  from  host 
computer  to  target  computer  when  they  are  physically  different 
units? 

15.  Have  the  security  requirements  of  multiple  projects  of  different 
security  levels  resident  on  the  computer  been  addressed? 

16.  What  facilities  (software  test  bed,  dynamic  simulation)  are 
proposed  to  support  the  testing  phases? 

17.  What  proprietary  data  rights  (if  any)  are  associated  with  each 
tool  within  the  environment? 

18.  By  what  procedures  could  the  government  acquire  these  tools? 

19.  By  what  procedures  are  these  tools  maintained?  Are  the  risks 
associated  with  outside  maintenance  adequately  addressed? 

20.  If  the  software  system  is  to  be  developed  on  the  contractor's 
environment  and,  upon  completion,  transferred  to  a  government 
installation,  has  sufficient  justification  been  provided  to 
ensure  that  such  a  transition  can  be  successfully  accomplished? 
Is  a  schedule  provided  to  accommodate  such  a  transfer  and,  if 
so,  is  it  realistic  within  the  scope  of  the  effort? 
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2.4  ENVIRONMENT 


Although  the  Ada  compiler  is  generally  regarded  as  the  most  critical 
support  software  tool  for  Ada  implementation,  there  are  many  other  software 
tools  in  existence  which  are  used  to  accomplish  a  software  development 
effort.  Such  tools  may  or  may  not  be  written  in  Ada.  This  report  does  not 
presume  that  an  Ada  Programming  Support  Environment  (APSE),  in  which  all 
tools  are  written  in  the  Ada  language  and  Interface  to  a  Kernel  APSE 
(KAPSE),  has  been  proposed  for  use.  The  following  questions  will  be  useful 
in  evaluating  the  suitability  of  the  proposed  environment  for  its  use  in 
the  overall  program. 

1.  What  environment  (support  software  tools)  is  proposed  for  use  in 
accomplishing  the  tasks  specified  within  the  RFP  Statement  of 
Work? 

2.  Is  the  environment: 

-  to  be  provided  as  Government  Furnished  Equlptment  (GFE)? 

-  to  be  delivered  to  the  government  upon  completion  of 
contract? 

-  contractor  owned/proprietary  from  which  software  must  be 
ported  to  a  government  environment  upon  completion  of 
contract? 

-  other  (specify)? 

3.  Reference  [12]  provides  a  detailed  taxonomy  of  software  tool 
features  for  a  life  cycle  software  engineering  environment. 

Which  of  the  features  enumerated  in  this  reference  are  provided 
by  the  tools  indicated  above? 

4.  What  features  other  than  those  identified  above  are  provided? 

5.  Does  the  environment  support  the  proposed  methodology? 

6.  What  is  the  level  of  Integration  of  the  tools? 

7.  What  is  the  maturity  level  of  the  environment;  i.e.,  have  the 
tools  been  used  on  other  Ada  software  development  programs  of 
comparable  level  of  effort? 

8.  Will  the  environment  allow  efficient  use  by  both  experts  and 
beginners;  for  example,  Is  prompting  via  menus  provided  for 
beginners  and  single  keystroke  capability  provided  for 
experienced  users? 
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-  input/output 

-  Interrupt  handling  (enable/disable/reenable)  from  the  Ada 
applications  program 

-  system  initialization 

-  system  services  callable  from  the  Ada  applications  program 

-  task  management 

*  abnormal  termination 

*  advanced  synchronization  and  communication 

*  attribute  access 

*  basic  synchronization  and  communication 

*  cyclic  tasking 

*  delay  and  time  services 

*  initiation 

*  interrupt  handling 

*  normal  termination 

*  scheduling 

-  type  and  data  conversion 

18.  Has  the  run  time  environment  been  stress  tested? 

19.  Does  the  run  time  environment  have  any  limitations  which  are  in 
conflict  with  the  mission  of  the  operational  program? 

20.  Is  the  performance  of  the  run  time  environment  sufficient  to 
support  the  performance  requirements  of  the  operational  program? 

21.  How  is  casualty  reconfiguration  of  the  operational  software 
handled? 

22.  Are  sizing  and  timing  studies  planned  throughout  the  software 
design  process  which  Include  the  evaluation  of  the  run  time 
operating  system  as  an  integral  part  of  the  system  design? 


8.  Has  a  backup  compiler  been  Identified  In  case  the  proposed 
compiler  cannot  be  used? 

9.  Has  the  proposed  Ada  compiler  been  validated?  If  not,  is 
rationale  provided  for  use  of  a  non-vali dated  compiler  and  is  an 
approach  for  transitioning  to  a  validated  compiler  outlined? 

10.  Have  the  Issues  and  risks  associated  with  periodic  compiler 
revalidation  been  Identified  and  addressed  with  contingency 
plans  proposed? 

11.  What  features  of  Chapter  13  of  MIL-STD-1815A  are  supported  by 
the  compiler?  Are  any  of  these  features  proposed  for  use  in  the 
implementation?  If  so,  has  the  Impact  of  using  such  features 
been  addressed? 

12.  What  features  of  Chapter  13  of  MIL-STD-1815A  are  not  supported 
by  the  compiler?  How  will  the  lack  of  such  features  impact  the 
project? 

13.  Has  the  proposed  compiler  previously  been  used  for  the  intended 
target  system? 

14.  On  what  other  programs  of  comparable  level  of  effort  has  the 
proposed  compiler  been  successfully  used?  Are  points  of  contact 
provided  to  enable  confirmation  of  this  information? 

15.  Is  there  an  existing  run  time  environment  for  the  target  system 
or  will  a  new  one  be  required?  If  one  exists: 

-  what  Is  Its  maturity  level 

-  will  modifications  be  required 

-  is  it  robust  enough  to  run  on  a  bare  target  machine 

16.  What  memory  resources  are  required  for  the  run  time  environment? 
Are  these  resource  requirements  reasonable  in  relation  to  the 
anticipated  memory  requirements  of  the  application  specified  In 
the  RFP  Statement  of  Work? 

17.  Is  specific  Information  provided  with  regard  to  how  the  run  time 
environment  supports  the  following  Ada  features  for  their 
suitability  with  the  application  specific  area  addressed  within 
the  RFP  Statement  of  Work: 

-  constraint  and  numeric  error  checking 

-  dynamic  storage  management 

-  exception  management 


2.3  IMPLEMENTATION  LANGUAGE 


If  Ada  has  been  proposed  as  the  implementation  language,  the  most 
immediate  aspects  associated  with  its  use  are:  (1)  the  guidelines  to  be  used 
by  programmers  during  software  development;  (2)  the  Ada  compiler  which  is  to 
be  used  during  the  effort;  and  (3)  the  run  time  support  environment  which 
will  be  used  on  the  target  machine.  The  assumption  which  is  made  in  this 
section  is  that  a  specific  compiler  has  not  been  mandated  in  the  RFP.  The 
following  questions  will  be  useful  in  evaluating  the  suitability  of  the 
proposed  guidelines,  compiler,  and  run  time  support  environment  for  their 
use  in  the  overall  program. 

1.  What  specific  guidelines  are  used  as  a  basis  for  implementation 
of  Ada?  Are  these  guidelines  consistent  with  the  software 
engineering  principles  which  are  promoted  through  use  of  the  Ada 
language?  (Example:  what  programming  standards  or  style  guides 
are  proposed?) 

2.  What  "checks  and  balances"  are  provided  to  ensure  compliance  to 
the  Ada  implementation  guidelines? 

3.  In  what  ways  will  the  proposed  implementation  exploit  the 
features  of  the  Ada  language? 

4.  Are  there  existing  software  components  proposed  for  inclusion 
within  the  program?  If  so,  what  rationale  is  provided  for  their 
use  and  by  what  procedure  will  these  existing  components  be 
incorporated  within  the  implementation?  Is  a  reusable  software 
library  available? 

5.  If  the  proposed  Ada  implementation  includes  the  import  of  code 
generated  by  another  programming  language,  has  the  proposal: 

-  provided  sufficient  rationale  for  the  inclusion  of  foreign 
code 

-  addressed  the  Impact  on  the  quality,  integrity,  and 
maintainability  of  the  final  product 

-  cited  previous  experience  with  foreign  code  import  for  use 
on  the  same  host  and  target  machines 

-  Identified  specific  software  development  tools  required 

6.  Are  performance  measurement  techniques  proposed  for  use  during 
the  Implementation?  If  so,  how  will  results  be  utilized? 

7.  What  is  the  availability  of  the  proposed  Ada  compiler?  If  the 
compiler  Is  not  currently  available,  but  is  or  will  be  under 
development,  has  a  risk  assessment  been  performed  as  to  the 
Impact  of  the  the  compiler  availability  on  the  overall  program? 


13.  What  configuration  control  procedures  are  used  for  the  PDL? 


14.  Will  the  PDL  code  be  delivered  as  part  of  the  product  source 
code  library? 

15.  Are  there  existing  PDL  components  proposed  for  inclusion  within 
the  design?  If  so,  what  rationale  is  provided  for  their  use  and 
by  what  procedure  will  these  existing  components  be  incorporated 
within  the  design?  Is  there  a  reusable  PDL  component  library 
available? 

16.  By  what  procedures  will  the  PDL  code  be  kept  up  to  date  once 
coding  has  begun? 

-  manual 

-  automated 

17.  On  what  other  programs  of  comparable  level  of  effort  has  the 
proposed  Ada-based  PDL  been  successfully  used?  Are  points  of 
contact  provi  'd  to  enable  confirmation  of  this  Information? 
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7.  For  preliminary  des.gn,  does  the  proposed  Ada-based  PDL  provide 
support  for: 

-  the  validation  of  preliminary  design  (where  "validation"  is 
defined  to  be  the  process  by  which  the  final  product  is 
ensured  to  satisfy  the  initial  specifications  and 
requirements) 

-  the  verification  of  preliminary  design  (where 
"verification"  is  defined  to  be  the  iterative  process  by 
which  the  progressively  more  detailed  product  fulfills  the 
requirements  levied  by  the  previous  step) 

-  capture  of  preliminary  design  information 

-  ease  of  change  to  the  preliminary  design 

-  ease  of  transition  to  the  detailed  design 

-  documentation  of  the  preliminary  design 

8.  For  detailed  design,  does  the  proposed  Ada-based  PDL  provide 
support  for: 

-  validation  of  detailed  design 

-  verification  of  detailed  design 

-  capture  of  detailed  design  information 

-  ease  of  change  to  the  detailed  design 

-  ease  of  transition  to  code 

-  documentation  of  the  detailed  design 

9.  Does  the  proposed  PDL  promote  design  portability  or  are  there 
hardware  dependencies  imposed  through  use  of  the  specific  PDL? 

10.  Is  there  a  clear,  demarcation  between  PDL  design  code  and  Ada 
Implementation  code? 

11.  What  automated  tools  support  the  proposed  PDL?  What  specific 
features  do  these  tools  provide  to  facilitate  the  design 
process? 

12.  What  documentation  is  provided  through  use  of  the  PDL  and  the 
PDL  specific  tools? 


2.2  PROGRAM  DESIGN  LANGUAGE 


A  POL  is  an  effective  mechanism  for  providing  an  accurate  description  of 
the  design  concepts.  Through  use  of  a  PDL,  the  design  of  the  overall  system 
is:  (1)  more  easily  understood  by  both  designers  and  nondesigners;  (2)  more 
readily  tracked  to  system  completion;  and  (3)  more  easily  modified  to 
accommodate  correction  of  deficiences  or  expansion  for  additional 
capabilities.  An  Ada-based  PDL  Is  one  which  is  based  upon  the  syntactic 
and  semantic  concepts  of  Ada  for  expression  of  the  design  concepts.  There 
currently  exists  within  the  Ada  community  many  Ada-based  PDLs,  some  of 
which  adhere  strictly  to  MIL-STD-1815A  and  some  of  which  allow  deviations 
from  the  standard.  The  following  questions  will  be  useful  in  evaluating 
the  suitability  of  the  proposed  Ada-based  PDL  for  its  use  in  the  overall 
program. 

1.  Has  a  specific  Ada-based  PDL  been  proposed?  If  so,  what 
supporting  rationale  has  been  provided  for  selection  of  this  PDL 
over  other  potential  PDLs?  If  not,  Is  there  a 
mapping/transition  path  defined  for  PDL  to  Ada  functionality? 

2.  What  specific  guidelines  are  used  as  a  basis  for  implementation 
of  the  proposed  PDL?  Are  these  guidelines  consistent  with  the 
software  engineering  principles  which  are  promoted  through  use 
of  the  Ada  language? 

3.  Is  the  proposed  Ada-based  PDL  implemented  through  manual  or 
automated  procedures? 

4.  Is  the  proposed  Ada-based  PDL  compilable? 

5.  What  "checks  and  balances"  are  provided  to  ensure  conformance  to 
the  Ada-based  PDL  guidelines? 

6.  Is  the  proposed  PDL  fully  compliant  with  the  Ada  language 
(MIL-STD-1815A)  or  does  the  PDL  support  deviations  from  the 
standard?  If  deviations  are  supported,  what  rationale  is 
provided  as  justification? 

-  the  Ada-based  PDL  Is  exactly  Ada 

-  the  Ada-based  PDL  Is  a  subset  of  Ada 

-  the  Ada-based  PDL  Is  a  subset  of  Ada  plus  extensions 

-  the  Ada-based  PDL  is  a  superset  of  Ada 


